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NATIONAL SECURITY COUNCIL 
WASHINGTON, D.C, 20506 


SECRET : August 9, 1972 


MEMORANDUM FOR RAY S. CLINE 


FROM: A, W. MARSHALL 


SUBJECT: Study Suggestions 


Ihave reviewed your three suggested studies (Chile, Yugoslavia, NSSM 108) 
and have discussed them with interested members of the NSC staff. Based 
upon the information now before me, I would prefer to see review efforts 
conducted towards Yugoslavia, Chile and NSSM 108 in that order. 


Ranking the three studies is a difficult task because they all have potential, 
However, [ do not favor starting with NSSM 108. Even if restricted to 
reviewing the management process, it would still be a major undertaking. 
Activities surrounding its development took nearly a year's time, To 
assess the management process would require investigating not only the 
link between policymakers and the community, but also the managerial 
efforts within the community to bring together analysts on a number of 
nations to develop a comprehensive picture of Latin America. 


| Chile and Yugoslavia seem more manageable and relevant to present needs. 
Chile and Allende obviously are important influences on U.S, policies to- 
wards Latin America; analysis better attuned to policymakers! needs would 
be helpful. Should the NSCIC Working Group chose this study, 1 recommend 
the time frame be moved back to January 1970 (or thereabouts) in order to 
evaluate the analytical output surrounding Allende's rise to power. The 
Yugoslavia study seems marginally more important than Chile because of 
the distinct possibility of political turmoil there in the near future, which 
could be exploited by other powers. 


Without going into detail on any one proposal, I have some general sugges- 


tions concerning the objectives, methodologies and the analytical or 
management issues to be addressed that may be of some use to yc. 
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OBJECTIVES 


The objectives of a study should be focused on presenting the NSCIC with 
lessons learned from reviewing the topic and developing specific guidance 
for the NSCIC to consider passing to the community. Detailed historical 
records will normally have to be compiled to carry out these studies, but 
this effort should not be allowed to overwhelm the study groups. Where 
necessary, sampling of products could be used, Also, as one of your 
studies suggests, comparing two years separated by four to five years 
might reduce the workload and focus on the question: --Have we improved 
over the last several years? What are the trends in key indicators of pro- 
duct quality/or analysis staff quality, etc.? 


What is important is (a) the evaluation of the intelligence process and 
products during the period, and (b) the presentation of a rigorously developed 
set of conclusions, general and specific recommendations, and appraisals of 
the likely impact of these recommendations on the quality of production and 
the internal functioning of the community. 


All three studies have a distinct management orientation in that they propose 
to examine the interplay among output, guidance received and resources 
available, Qualitative evaluation of a particular product apparently would 
center on asking whether it clearly made its points and whether these points 
were relevant to the needs of policymakers at the time. Little work will be 
performed directly to evaluate the intrinsic quality of the product -- i.e., 
whether it represented an exhaustive perceptive analysis -- except as this 
falls out from the study. This approach does not disturb me... I find it 
perfectly appropriate to target the study efforts on the management issues 
as well as on analytical ones. However, for some selected products an 
assessment of the quality of the intelligence analysis, especially as regards. 
political and governmental behavior, would be of interest. What is impor- 
tant is to insure that everyone participating in the study understands the 
approach, 


METHODS 


I envision that the role of the working level participants in a study would go 
beyond that of recognizing problems and making general recommendations. 
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Since these individuals will be the ones most familiar with the intelligence 
effort surrounding the topic, and will have spent the greatest time thinking 
about it, they should have some valuable thoughts about both the very broad 
changes needed and the specific adjustments which could be made in the 
present system of managing the intelligence process, Fully exploiting 
their insights is essential to drawing maximum value from the work which 
goes into studies, The working group can still be the final judge of the 
weight of the case the study group makes. 


I suggest the NSCIC Working Group select one of the topics to begin immedi- 
ate work on. You should guide the study and I would be delighted to help you 
in any way I can, The remaining two could be postponed until later in the 
year when present studies will be nearing completion. At that time these 
topics could be considered along with any others that have been submitted. 


Past studies have experimented with panels, individual analysis and group 
reports, and some of these methods may not be appropriate to the proposed 
three topics, For example, a panel may be of little value in evaluating a 
management process if its members are unfamiliar with the community; 
however, an appropriately selected panel could be very helpful in judging 
whether a product presented a clear message to the policymaker. If the 
management process is to be a key area, a working group should also 
consider using management ovriented individuals (such as OMB personnel) 
who have no vested interests. 


~ 


MANAGEMENT AND ANALYTICAL ISSUES 


While the issues outlined in the scope of each of your proposals give a broad 
view of each study's direction, a more detailed list of issues should be for- 
mulated before a study actually begins. The following are some suggestions. 


Study Execution 


Prior to beginning the compilation of information I think a study group 
should reach some tentative ideas about the scope of their effort. Any 
pre-study theories should be discussed to decide how to test for them, 
The time period and the actual intelligence products to be covered (e.g., 
NIE's, daily briefings, etc.) should be laid out with special attention to 
what is being ignored and why. A general format for the report should 
be discussed. 
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Management Issues 


An excellent first step to understanding the management of intelligence 
during a period would be to study the linkage along which information 
flowed within the community and between the community and the policy- 
makers, Both the informal and formal aspects of these links could be 
examined, Attention could be given to the flow of information, feedback 
and guidance, and to how the flow was affected by the perceptions of dif- 
ferent participants (either agencies or individuals), by time constraints, 
and by procedures. However, I recognize that to develop such a detailed 
perspective is extremely difficult when events occurred months in the past, 
people have changed, etc. 


A particularly interesting issue would be to’ identify the implicit standards 
which were set for intelligence output (e. g., number of reports, response 
time to questions, etc.) and how the community measured up to those 
criteria, Furthermore, who set the criteria and why? 


Consideration should be given to how the output of the community could 

have been improved through better management. In the short term this 
includes examining the role of middle managers in motivating analysts, in 
communicating needs to analysts, and in matching the analysts! effort with 
the needs of the consumers, The role of higher level intelligence managers 
shouldbe subjected to the same review. (In the long term the working group 
may become interested in such questions as how training influenced the roles 
of both analyst and manager and how the system of personnel management 
and motivation affect their behavior. ) 


Analytical Issues 


The "quality? of analysis is a frequently discussed topic. In addressing it 
an effort must be made to define what we mean by quality -- is ita rigorous 
analysis, one that combines all viewpoints, one that reads well, etc. The 
criteria of quality may vary markedly between individuals and agencies; in 
fact the lack of consensus could well be a major part of the problem. 


Examining the analytical output requires an appreciation for the raw data 
being gathered aud the validity being implicitly given to it (simply because 
we collect it covertly does not mean it is egeurates regardless of the 


SECRET 
Approved For Release 2001/08/14 : CIA-RDP84B00506R000100050014-5 


Approved For Relegse 2001/08/14 : CIA-RDP84BO00506R0Q)/ 00050014-5 


SECRET 


collection device). The modification in the raw data as it is processed 
may merit review in order to see how it influences the analyst. The 
analyst's work is then perhaps the key to the quality of the product. How 
does he assemble data? What methods does he use to analyze it -- does 
he just assemble data and make obvious inferences; does he test different 
analytical frameworks against the data; what data does he dismiss? In 
looking at the final written product one needs to consider whether major 
points are clear, whether they respond to consumer needs, and how the 
analysts try to communicate the appropriate level of uncertainty. 
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